Search for: All records

We formulate a unifying framework for *unsupervised continual learning (UCL)*, which disentangles learning objectives that are specific to the present and the past data, encompassing *stability*, *plasticity*, and *cross-task consolidation*. The framework reveals that many existing UCL approaches overlook cross-task consolidation and try to balance plasticity and stability in a shared embedding space. This results in worse performance due to a lack of within-task data diversity and reduced effectiveness in learning the current task. Our method, *Osiris*, which explicitly optimizes all three objectives on separate embedding spaces, achieves state-of-the-art performance on all benchmarks, including two novel ones proposed in this paper featuring semantically structured task sequences. Finally, we show some preliminary evidence that continual models can benefit from such more realistic learning scenarios. 

Large-scale deep neural networks (DNNs) are both compute and memory intensive. As the size of DNNs continues to grow, it is critical to improve the energy efficiency and performance while maintaining accuracy. For DNNs, the model size is an important factor affecting performance, scalability and energy efficiency. Weight pruning achieves good compression ratios but suffers from three drawbacks: 1) the irregular network structure after pruning, which affects performance and throughput; 2) the increased training complexity; and 3) the lack of rigirous guarantee of compression ratio and inference accuracy. To overcome these limitations, this paper proposes CirCNN, a principled approach to represent weights and process neural networks using block-circulant matrices. CirCNN utilizes the Fast Fourier Transform (FFT)-based fast multiplication, simultaneously reducing the computational complexity (both in inference and training) from O(n2) to O(n log n) and the storage complexity from O(n2) to O(n), with negligible accuracy loss. Compared to other approaches, CirCNN is distinct due to its mathematical rigor: the DNNs based on CirCNN can converge to the same "effectiveness" as DNNs without compression. We propose the CirCNN architecture, a universal DNN inference engine that can be implemented in various hardware/software platforms with configurable network architecture (e.g., layer type, size, scales, etc.). In CirCNN architecture: 1) Due to the recursive property, FFT can be used as the key computing kernel, which ensures universal and small-footprint implementations. 2) The compressed but regular network structure avoids the pitfalls of the network pruning and facilitates high performance and throughput with highly pipelined and parallel design. To demonstrate the performance and energy efficiency, we test CirCNN in FPGA, ASIC and embedded processors. Our results show that CirCNN architecture achieves very high energy efficiency and performance with a small hardware footprint. Based on the FPGA implementation and ASIC synthesis results, CirCNN achieves 6 - 102X energy efficiency improvements compared with the best state-of-the-art results. 

Abstract First reported in the 1960s, offshore freshened groundwater (OFG) has now been documented in most continental margins around the world. In this review we compile a database documenting OFG occurrences and analyze it to establish the general characteristics and controlling factors. We also assess methods used to map and characterize OFG, identify major knowledge gaps, and propose strategies to address them. OFG has a global volume of 1 × 10⁶ km³; it predominantly occurs within 55 km of the coast and down to a water depth of 100 m. OFG is mainly hosted within siliciclastic aquifers on passive margins and recharged by meteoric water during Pleistocene sea level lowstands. Key factors influencing OFG distribution are topography‐driven flow, salinization via haline convection, permeability contrasts, and the continuity/connectivity of permeable and confining strata. Geochemical and stable isotope measurements of pore waters from boreholes have provided insights into OFG emplacement mechanisms, while recent advances in seismic reflection profiling, electromagnetic surveying, and numerical models have improved our understanding of OFG geometry and controls. Key knowledge gaps, such as the extent and function of OFG, and the timing of their emplacement, can be addressed by the application of isotopic age tracers, joint inversion of electromagnetic and seismic reflection data, and development of three‐dimensional hydrological models. We show that such advances, combined with site‐specific modeling, are necessary to assess the potential use of OFG as an unconventional source of water and its role in sub‐seafloor geomicrobiology.